k8s pod slave 集成 Jenkinsfile

插件官方文档

  1. 对接 kubernetes 集群
  2. 使用 kubernetes 的 Pod-Template 来作为动态的 agent 执行 Jenkins 任务
  3. 如何制作 agent 容器实现不同类型的业务的集成
  4. 集成代码扫描、docker镜像自动构建、k8s服务部署、自动化测试

jenkins 集成 Kubernetes

插件安装及配置 (普通版本)jenkins 在 eks 集群中部署时

插件官方文档

  1. [系统管理] -> [插件管理] -> [搜索kubernetes]->直接安装 若安装失败,请先更新 bouncycastle API Plugin并重新启动Jenkins

    image-20220726170827529

  2. [系统管理] -> [系统配置] -> [Add a new cloud]

    image-20220726171140172
  3. 配置地址信息

  4. 配置Pod Template

    • 名称:jnlp-slave
    • 命名空间:jenkins
    • Jenkins 通道 :jenkins:50000
    • 标签列表:jnlp-slave,作为agent的label选择用
    • 连接 Jenkins 的超时时间(秒) :300,设置连接jenkins超时时间
    • 节点选择器:agent=true
    • 工作空间卷:选择hostpath,设置/opt/jenkins_jobs/,注意需要设置chown -R 1000:1000 /opt/jenkins_jobs/权限,否则Pod没有权限

jenkins 添加 k8s

1、添加凭据

Jenkins 添加一个连接 kubernetes 集群的凭据。

1)类型:Secret text
2)范围:全局
3)Secret:confi文件中的token值
4)id:kube(自定义命名)

2、配置kubernetes

Jenkins 系统设置 --> 最下面,增加一个云(kubernetes)

1)名称:kubernetes
2)Kubernetes地址:config文件中的server值,https://rancher.xxxxxx.com/k8s/clusters/c-xg99q
3)Kubernetes服务证书key:config文件中的certificate-authority-data值,但是这个值需要进行转换,命令:echo xxxxxxxxx | base64 -d >ca.crt
4)凭据:之前创建的凭据
5)Jenkins地址:http://jenkinscicd.xxxxxx.com

jenkins 在 eks 集群之外部署时,连接 eks

eks 之外的 jenkins 连接到 eks ,这个过程还是比较麻烦的。它涉及到 aws 的认证 + eks 的认证;

过程大概分为以下步骤:

  1. 首先需要在 eks 所在 的 aws 账号创建 iam 用户
  2. 授权iam用户对eks有操作权限,最小的权限为可读权限;
  3. 绑定 iam 用户到 eks 中的用户
  4. eks 用户 绑定 role 来分配权限;
  5. jenkins 机器配置 aws cli
  6. aws cli 更新 kubeconfig
  7. 将kubeconfig 导入jenkins,并测试 jenkins 连接 eks
  8. 配置 jenkins anget 端口(系统管理->全局安全配置->代理->指定端口(50000),可以这样配置。),然后jenkins kubenetes 中配置jenkins 连接地址为:http://<jenkins master 域名地址>:8080(实际测试用其他端口失败,包括nginx代理的8080,正在查找错误原因)
  9. 接下来就可以配置一个测试项目,测试 jenkins pipeline 构建了;

详情请参考链接

eks 线上 jenkins pipeline 配置

  • 该项目中我们在一个pod中包含了多个容器
    • jnlp 或者 jenkins/jnlp-slave:4.0.1-1 : jnlp 容器只是负责和 jenkins master 建立连接;
    • maven:3.8.1-jdk-8:负责打包
    • docker:负责构建镜像,推送镜像
    • awscli:负责获取 ecr 登录密钥;
  • 我们在pod中绑定了 serviceAccount:jenkins-agentd-pod-service-account,它提供 ecr 所必需的权限
  • 我们指定了工作目录,workingDir: '/home/jenkins/agent',
  • 指定了 runAsUser :0;因为默认 jenkins-agent 启动权限为1000,不然还得修改挂载目录的权限,这样就不用修改目录权限了
  • 我们共享了 jenkins-agent-pvc:/root/.m2/repository 目录挂载,该文件中存有 maven 所依赖的包下载,挂载出来避免重复下载
  • 我们配置了 maven-config:/root/.m2;将maven 的配置以 configmap 方式挂载出来,方便后期管理;
  • 该任务中并没有配置 k8s 的部署,后期会加入;
// 之前的项目较老。所以一些配置使用的比较原始
podTemplate( serviceAccount:'jenkins-agentd-pod-service-account',
    containers: [
        containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:latest', workingDir: '/home/jenkins/agent', privileged: 'true', runAsUser: '0'),
        containerTemplate(name: 'maven', image: 'maven:3.8.1-jdk-8', ttyEnabled: true, command: 'cat', workingDir: '/home/jenkins/agent', privileged: 'true', runAsUser: '0'),
        containerTemplate(name: 'docker', image: 'docker', ttyEnabled: true, command: 'cat', workingDir: '/home/jenkins/agent', privileged: 'true', runAsUser: '0'),
        containerTemplate(name: 'awscli', image: 'amazon/aws-cli', ttyEnabled: true, command: 'cat', workingDir: '/home/jenkins/agent', privileged: 'true', runAsUser: '0'),
    ],
    volumes: [
        // # hostPathVolume(hostPath: '/tmp/maven/repository', mountPath:'/root/.m2/repository'),
        persistentVolumeClaim(claimName: 'jenkins-agent-pvc', mountPath: '/root/.m2/repository'),
        hostPathVolume(hostPath: '/var/run/docker.sock', mountPath:'/var/run/docker.sock'),
        hostPathVolume(hostPath: '/etc/docker/daemon.json', mountPath:'/etc/docker/daemon.json'),
        configMapVolume(configMapName: 'maven-config', mountPath:'/root/.m2')
    ],
  )
  {

    node(POD_LABEL) {

        stage('拉取 git仓库 代码') {
            // 取消 ssl 检查
            sh 'git config --global http.sslverify false'
            sh 'git config --global https.sslverify false'
            git credentialsId: '6df8b3dc-bb50-4464-8cc6-ae58b9148f66', url: 'https://gitea.nqspace.com/hanqunfeng/lx-novel.git'

        }

        stage('maven 打包') {
            container('maven') {
                stage('构建 Maven 项目') {
                    sh 'mvn -version'
                    sh 'mvn clean package -Dmaven.test.skip=true'
                }
            }
        }

        stage('构建docker镜像 grap_novel_data') {

            container('awscli') {
                stage('获取ecr登陆密钥') {
                    sh '''
                        pwd
                        aws ecr get-login-password --region us-west-2 > dockerLogin
                        cp dockerLogin novel-web-parent/grap_novel_data/target/
                        cp dockerLogin novel-web-parent/novel-boss/target/
                        cp dockerLogin novel-web-parent/novel-boss/target/
                    '''
                }
            }

            container('docker') {
                stage('build docker') {
                    sh 'pwd'
                    dir('novel-web-parent/grap_novel_data/target') {
                        sh '''
                            pwd
                            sh docker-push.sh
                        '''
                    }
                }
            }
        }

        stage('构建docker镜像 novel-boss') {
            container('docker') {
                stage('build docker') {
                    sh 'pwd'
                    dir('novel-web-parent/novel-boss/target') {
                        sh '''
                            pwd
                            sh docker-push.sh
                        '''
                    }
                }
            }
        }

        stage('构建docker镜像 novel-api') {
            container('docker') {
                stage('build docker') {
                    sh 'pwd'
                    dir('novel-web-parent/novel-api/target') {
                        sh '''
                            pwd
                            sh docker-push.sh
                        '''
                    }
                }
            }
        }
    }
}

Pod-Template 中容器镜像的制作

制作一个tools镜像,集成常用的工具,来完成常见的构建任务,需要注意的几

点:

  • 使用alpine基础镜像,自身体积比较小
  • 替换国内安装源
  • 为了使用docker,安装了docker
  • 为了克隆代码,安装git
  • 为了后续做python的测试等任务,安装python环境
  • 为了后面集成robotframework,安装了chromium-chromedriver等包
  • 为了在容器中调用 kubectl 的命令,拷贝了kubectl的二进制文件
  • 为了认证 kubectl,需要在容器内部生成 .kube 目录及 config 文件
$ mkdir tools;
$ cp `which kubectl` .
$ cp ~/.kube/config .
$ cat requirements.txt 
robotframework
robotframework-seleniumlibrary
robotframework-databaselibrary
robotframework-requests

Dockerfile

jenkins/custom-images/tools/Dockerfile

FROM alpine
LABEL maintainer="inspur_lyx@hotmail.com"
USER root
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.tuna.tsinghua.edu.cn/g'
/etc/apk/repositories && \
    apk update && \
    apk add  --no-cache openrc docker git curl tar gcc g++ make \
    bash shadow openjdk8 python python-dev py-pip openssl-dev libffi-dev \
    libstdc++ harfbuzz nss freetype ttf-freefont chromium chromium-chromedriver
&& \
    mkdir -p /root/.kube && \
    usermod -a -G docker root
COPY requirements.txt /
COPY config /root/.kube/
RUN pip install -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host
mirrors.aliyun.com -r /requirements.txt
RUN rm -rf /var/cache/apk/* && \
    rm -rf ~/.cache/pip
#-----------------安装 kubectl--------------------# COPY kubectl /usr/local/bin/
RUN chmod +x /usr/local/bin/kubectl
# ------------------------------------------------#

执行镜像构建并推送到仓库中

$ docker build . -t 172.21.32.13:5000/devops/tools:v1
$ docker push 172.21.32.13:5000/devops/tools:v1

我们可以直接使用该镜像做测试:

## 启动临时镜像做测试
$ docker run --rm -ti 172.21.32.13:5000/devops/tools:v1 bash # / git clone http://xxxxxx.git
# / kubectl get no
# / python3
#/ docker
## 重新挂载docker的sock文件
docker run -v /var/run/docker.sock:/var/run/docker.sock --rm -ti 172.21.32.13:5000/devops/tools:v1 bash

更新Jenkins中的 PodTemplate,添加tools镜像,注意同时要先添加名为jnlp的container,因为我们是 使用自定义的PodTemplate覆盖掉默认的模板:

image-20220728121009770

在卷栏目,添加卷,Host Path Volume,不然在容器中使用docker会提示docker服务未启动

image-20220728121027153

实践通过Jenkinsfile实现demo项目自动发布到kubenetes环境

tools容器做好后,我们需要对Jenkinsfile做如下调整:

jenkins/pipelines/p8.yaml

pipeline {
    agent { label 'jnlp-slave'}

    options {
    	buildDiscarder(logRotator(numToKeepStr: '10'))
    	disableConcurrentBuilds()
    	timeout(time: 20, unit: 'MINUTES')
    	gitLabConnection('gitlab')
    }

    environment {
        IMAGE_REPO = "172.21.32.13:5000/myblog"
        DINGTALK_CREDS = credentials('dingTalk')
        TAB_STR = "\n                    \n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"
    }

    stages {
        stage('printenv') {
            steps {
                script{
                    sh "git log --oneline -n 1 > gitlog.file"
                    env.GIT_LOG = readFile("gitlog.file").trim()
                }
                sh 'printenv'
            }
        }
        stage('checkout') {
            steps {
                container('tools') {
                    checkout scm
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS = env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('build-image') {
            steps {
                container('tools') {
                    retry(2) { sh 'docker build . -t ${IMAGE_REPO}:${GIT_COMMIT}'}
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('push-image') {
            steps {
                container('tools') {
                    retry(2) { sh 'docker push ${IMAGE_REPO}:${GIT_COMMIT}'}
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('deploy') {
            steps {
                container('tools') {
                    sh "sed -i 's#{{IMAGE_URL}}#${IMAGE_REPO}:${GIT_COMMIT}#g' deploy/*"
                    timeout(time: 1, unit: 'MINUTES') {
                        sh "kubectl apply -f deploy/"
                    }
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
    }
    post {
        success { 
            echo 'Congratulations!'
            sh """
                curl 'https://oapi.dingtalk.com/robot/send?access_token=${DINGTALK_CREDS_PSW}' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "title":"myblog",
                            "text": "😄👍 构建成功 👍😄  \n**项目名称**:luffy  \n**Git log**: ${GIT_LOG}   \n**构建分支**: ${BRANCH_NAME}   \n**构建地址**:${RUN_DISPLAY_URL}  \n**构建任务**:${BUILD_TASKS}"
                        }
                    }'
            """ 
        }
        failure {
            echo 'Oh no!'
            sh """
                curl 'https://oapi.dingtalk.com/robot/send?access_token=${DINGTALK_CREDS_PSW}' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "title":"myblog",
                            "text": "😖❌ 构建失败 ❌😖  \n**项目名称**:luffy  \n**Git log**: ${GIT_LOG}   \n**构建分支**: ${BRANCH_NAME}  \n**构建地址**:${RUN_DISPLAY_URL}  \n**构建任务**:${BUILD_TASKS}"
                        }
                    }'
            """
        }
        always { 
            echo 'I will always say Hello again!'
        }
    }
}

集成sonarQube实现代码扫描

sonarqube架构简介

image-20220728141431525

  1. SonarQube Scanner 扫描仪在本地执行代码扫描任务
  2. 执行完后,将分析报告被发送到SonarQube服务器进行处理
  3. SonarQube服务器处理和存储分析报告导致SonarQube数据库,并显示结果在UI中。

sonarqube on kubernetes环境搭建

  1. 资源文件准备

sonar/secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: sonar
  namespace: jenkins
type: Opaque
data:
  POSTGRES_USER: cm9vdA==	# root
  POSTGRES_PASSWORD: MTIzNDU2	# 123456

sonar/postgres.yaml

apiVersion: v1
kind: Service
metadata:
  name: sonar-postgres
  labels:
    app: sonar-postgres
  namespace: jenkins
spec:
  ports:
  - name: server
    port: 5432
    targetPort: 5432
    protocol: TCP
  selector:
    app: sonar-postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: jenkins
  name: sonar-postgres
  labels:
    app: sonar-postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sonar-postgres
  template:
    metadata:
      labels:
        app: sonar-postgres
    spec:
      nodeSelector:
        sonar: "true"
      tolerations:
      - operator: "Exists"
      containers:
      - name: postgres
        image:  172.21.32.13:5000/postgres:11.4
        imagePullPolicy: "IfNotPresent"
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB             #PostgreSQL 数据库名称
          value: "sonar"
        - name: POSTGRES_USER           #PostgreSQL 用户名
          valueFrom:
            secretKeyRef:
              name: sonar
              key: POSTGRES_USER
        - name: POSTGRES_PASSWORD       #PostgreSQL 密码
          valueFrom:
            secretKeyRef:
              name: sonar
              key: POSTGRES_PASSWORD
        resources:
          limits:
            cpu: 1000m
            memory: 2048Mi
          requests:
            cpu: 500m
            memory: 1024Mi
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgredb
      volumes:
      - name: postgredb
        hostPath:
          path: /var/lib/postgres/
          type: Directory

sonar/sonar.yaml

apiVersion: v1
kind: Service
metadata:
  name: sonarqube
  namespace: jenkins
  labels:
    app: sonarqube
spec:
  ports:
  - name: sonarqube
    port: 9000
    targetPort: 9000
    protocol: TCP
  selector:
    app: sonarqube
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: jenkins
  name: sonarqube
  labels:
    app: sonarqube
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sonarqube
  template:
    metadata:
      labels:
        app: sonarqube
    spec:
      nodeSelector:
        sonar: "true"
      initContainers:
      - command:
        - /sbin/sysctl
        - -w
        - vm.max_map_count=262144
        image: alpine:3.6
        imagePullPolicy: IfNotPresent
        name: elasticsearch-logging-init
        resources: {}
        securityContext:
          privileged: true
      containers:
      - name: sonarqube
        image: 172.21.32.13:5000/sonarqube:7.9-community
        ports:
        - containerPort: 9000
        env:
        - name: SONARQUBE_JDBC_USERNAME
          valueFrom:
            secretKeyRef:
              name: sonar
              key: POSTGRES_USER
        - name: SONARQUBE_JDBC_PASSWORD
          valueFrom:
            secretKeyRef:
              name: sonar
              key: POSTGRES_PASSWORD
        - name: SONARQUBE_JDBC_URL
          value: "jdbc:postgresql://sonar-postgres:5432/sonar"
        livenessProbe:
          httpGet:
            path: /sessions/new
            port: 9000
          initialDelaySeconds: 60
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /sessions/new
            port: 9000
          initialDelaySeconds: 60
          periodSeconds: 30
          failureThreshold: 6
        resources:
          limits:
            cpu: 2000m
            memory: 4096Mi
          requests:
            cpu: 300m
            memory: 512Mi
      volumes:
      - name: sonarqube-data
        hostPath:
          path: /opt/sonarqube/data
      - name: sonarqube-logs
        hostPath:
          path: /opt/sonarqube/logs
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: sonarqube
  namespace: jenkins
spec:
  rules:
  - host: sonar.qq.com
    http:
      paths:
      - backend:
          serviceName: sonarqube
          servicePort: 9000
        path: /
status:
  loadBalancer: {}
## 创建secret
$ kubectl create -f secret.yaml
## 打算将sonarqube部署在哪个节点,打上label,sonar=true $ kubectl label node k8s-master sonar=true
## 到sonar=true的节点创建postgres目录
$ mkdir /var/lib/postgres/
# 创建postgres数据库
$ kubectl create -f postgres.yaml
## 创建sonarqube服务器
$ kubectl create -f sonar.yaml
## 配置本地hosts解析 152.136.62.143 sonar.qq.com
## 访问sonarqube,初始用户名密码为 admin/admin http://sonar.qq.com
  1. sonar-scanner的安装

下载地址: https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-c li-4.2.0.1873-linux.zip。该地址比较慢,可以在网盘下载(https://pan.baidu.com/s/1SiEhWyHikTiKl5lEMX1tJg 提取码: tqb9)。

  1. 演示sonar代码扫描功能

    • 在项目根目录中准备配置文件 sonar-project.properties

      sonar.projectKey=myblog
      sonar.projectName=myblog
      # if you want disabled the DTD verification for a proxy problem for
      example, true by default
      sonar.coverage.dtdVerification=false
      # JUnit like test report, default value is test.xml
      sonar.sources=.
      
    • 配置sonarqube服务器地址

      由于sonar-scanner需要将扫描结果上报给sonarqube服务器做质量分析,因此我们需要在

      sonar-scanner中配置sonarqube的服务器地址:

      在宿主机中测试,可以先用cluster-ip,先查看一下sonarqube的cluster-ip:

      $ kubectl -n jenkins get svc|grep sonarqube
      sonarqube        ClusterIP   10.105.94.31    <none>        9000/TCP
      $ cat sonar-scanner/conf/sonar-scanner.properties
      #----- Default SonarQube server
      #sonar.host.url=http://localhost:9000
      sonar.host.url=http://10.105.94.31:9000
      #----- Default source code encoding
      #sonar.sourceEncoding=UTF-8
      
    • 执行扫描

      ## 在项目的根目录下执行
      $ /opt/sonar-scanner-4.0.0.1744-linux/bin/sonar-scanner -X
      
    • sonarqube界面查看结果

      登录sonarqube界面查看结果,Quality Gates说明

插件安装及配置

  1. 集成到tools容器中

    由于我们的代码拉取、构建任务均是在tools容器中进行,因此我们需要把scanner集成到我们的 tools容器中,又因为scanner是一个cli客户端,因此我们直接把包解压好,拷贝到tools容器内 部,配置一下PATH路径即可,注意两点:

    • 由于是在k8s集群中利用Pod去执行扫描任务,因此可以直接调用http://sonarqube:9000%E8%BF%9E 接服务端

    • 由于tools已经集成了java环境,因此可以直接剔除scanner自带的jre

      • 删掉sonar-scanner/jre目录

      • 修改sonar-scanner/bin/sonar-scanner

        use_embedded_jre=false

      $ cd tools
      $ cp -r /opt/sonar-scanner-4.0.0.1744-linux/ sonar-scanner ## sonar配置,由于我们是在Pod中使用,因此可以直接配置: sonar.host.url=http://sonarqube:9000
      $ cat sonar-scanner/conf/sonar-scanner.properties
      #----- Default SonarQube server sonar.host.url=http://sonarqube:9000
      #----- Default source code encoding
      #sonar.sourceEncoding=UTF-8
      $ rm -rf sonar-scanner/jre
      $ vi sonar-scanner/bin/sonar-scanner
      ...
      use_embedded_jre=false
      ...
      

      Dockerfile

      jenkins/custom-images/tools/Dockerfile2

      FROM alpine
      LABEL maintainer="inspur_lyx@hotmail.com"
      USER root
      
      RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apk/repositories && \
          apk update && \
          apk add  --no-cache openrc docker git curl tar gcc g++ make \
          bash shadow openjdk8 python python-dev py-pip openssl-dev libffi-dev \
          libstdc++ harfbuzz nss freetype ttf-freefont chromium chromium-chromedriver && \
          mkdir -p /root/.kube && \
          usermod -a -G docker root
      
      COPY requirements.txt /
      COPY config /root/.kube/
      
      RUN pip install -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /requirements.txt
      
      RUN rm -rf /var/cache/apk/* && \
          rm -rf ~/.cache/pip
      #-----------------安装 kubectl--------------------#
      COPY kubectl /usr/local/bin/
      RUN chmod +x /usr/local/bin/kubectl
      # ------------------------------------------------#
      
      #---------------安装 sonar-scanner-----------------#
      COPY sonar-scanner /usr/lib/sonar-scanner
      RUN ln -s /usr/lib/sonar-scanner/bin/sonar-scanner /usr/local/bin/sonar-scanner && chmod +x /usr/local/bin/sonar-scanner
      ENV SONAR_RUNNER_HOME=/usr/lib/sonar-scanner
      # ------------------------------------------------#
      

      重新构建镜像,并推送到仓库:

      $ docker build . -t 172.21.32.13:5000/devops/tools:v2
      $ docker push 172.21.32.13:5000/devops/tools:v2
      
  2. 修改Jenkins PodTemplate 为了在新的构建任务中可以拉取v2版本的tools镜像,需要更新PodTemplate

  3. 安装并配置sonar插件

    由于sonarqube的扫描的结果需要进行Quality Gates的检测,那么我们在容器中执行完代码扫描 任务后,如何知道本次扫描是否通过了Quality Gates,那么就需要借助于sonarqube实现的 jenkins的插件。

    • 安装插件

      插件中心搜索sonarqube,直接安装

    • 配置插件

      系统管理->系统配置-> SonarQube servers ->Add SonarQube

      1. 登录sonarqube -> My Account -> Security -> Generate Token
      2. 登录Jenkins,添加全局凭据,类型为Secret text
    • 如何在jenkinsfile中使用

      我们在 https://jenkins.io/doc/pipeline/steps/sonar/ 官方介绍中可以看到:

      stage('build && SonarQube analysis') {
          steps {
              withSonarQubeEnv('sonarqube') {
                  sh 'sonar-scanner -X'
              }
          }
      }
      stage("Quality Gate") {
          steps {
              timeout(time: 1, unit: 'HOURS') {
                  // Parameter indicates whether to set pipeline to UNSTABLE if Quality Gate fails
                  // true = set pipeline to UNSTABLE, false = don't
              }
          }
      }
      

Jenkinsfile集成sonarqube演示

jenkins/pipelines/p9.yaml

pipeline {
    agent { label 'jnlp-slave'}

    options {
    	buildDiscarder(logRotator(numToKeepStr: '10'))
    	disableConcurrentBuilds()
    	timeout(time: 20, unit: 'MINUTES')
    	gitLabConnection('gitlab')
    }

    environment {
        IMAGE_REPO = "172.21.32.13:5000/demo/myblog"
        DINGTALK_CREDS = credentials('dingTalk')
        TAB_STR = "\n                    \n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"
    }

    stages {
        stage('git-log') {
            steps {
                script{
                    sh "git log --oneline -n 1 > gitlog.file"
                    env.GIT_LOG = readFile("gitlog.file").trim()
                }
                sh 'printenv'
            }
        }        
        stage('checkout') {
            steps {
                container('tools') {
                    checkout scm
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS = env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('CI'){
            failFast true
            parallel {
                stage('Unit Test') {
                    steps {
                        echo "Unit Test Stage Skip..."
                    }
                }
                stage('Code Scan') {
                    steps {
                        container('tools') {
                            withSonarQubeEnv('sonarqube') {
                                sh 'sonar-scanner -X'
                                sleep 3
                            }
                            script {
                                timeout(1) {
                                    def qg = waitForQualityGate('sonarqube')
                                    if (qg.status != 'OK') {
                                        error "未通过Sonarqube的代码质量阈检查,请及时修改!failure: ${qg.status}"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
        stage('build-image') {
            steps {
                container('tools') {
                    retry(2) { sh 'docker build . -t ${IMAGE_REPO}:${GIT_COMMIT}'}
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('push-image') {
            steps {
                container('tools') {
                    retry(2) { sh 'docker push ${IMAGE_REPO}:${GIT_COMMIT}'}
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
        stage('deploy') {
            steps {
                container('tools') {
                    sh "sed -i 's#{{IMAGE_URL}}#${IMAGE_REPO}:${GIT_COMMIT}#g' deploy/*"
                    timeout(time: 1, unit: 'MINUTES') {
                        sh "kubectl apply -f deploy/"
                    }
                }
                updateGitlabCommitStatus(name: env.STAGE_NAME, state: 'success')
                script{
                    env.BUILD_TASKS += env.STAGE_NAME + "√..." + env.TAB_STR
                }
            }
        }
    }
    post {
        success { 
            echo 'Congratulations!'
            sh """
                curl 'https://oapi.dingtalk.com/robot/send?access_token=${DINGTALK_CREDS_PSW}' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "title":"myblog",
                            "text": "😄👍 构建成功 👍😄  \n**项目名称**:luffy  \n**Git log**: ${GIT_LOG}   \n**构建分支**: ${BRANCH_NAME}   \n**构建地址**:${RUN_DISPLAY_URL}  \n**构建任务**:${BUILD_TASKS}"
                        }
                    }'
            """ 
        }
        failure {
            echo 'Oh no!'
            sh """
                curl 'https://oapi.dingtalk.com/robot/send?access_token=${DINGTALK_CREDS_PSW}' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "title":"myblog",
                            "text": "😖❌ 构建失败 ❌😖  \n**项目名称**:luffy  \n**Git log**: ${GIT_LOG}   \n**构建分支**: ${BRANCH_NAME}  \n**构建地址**:${RUN_DISPLAY_URL}  \n**构建任务**:${BUILD_TASKS}"
                        }
                    }'
            """
        }
        always { 
            echo 'I will always say Hello again!'
        }
    }
}

集成RobotFramework实现验收测试

链接地址;使用谷歌浏览器打开,有目录;

代码地址:cicd: cicd 配置 (gitee.com)

基于sharedLibrary进行CI/CD流程的优化

链接地址

代码地址:cicd: cicd 配置 (gitee.com)